Papers with syntactic language model
Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale (2024.acl-long)
Copied to clipboard
| Challenge: | Existing syntactic language models require a gold tree and sequential training to generate sentences. |
| Approach: | They propose an unsupervised syntactic language model that incrementally generates a sentence with its syntaktic tree in a left-to-right manner. |
| Outcome: | The proposed model outperforms existing models on grammar induction and comprehension tasks while holding a substantial acceleration on training. |
What represents “style” in authorship attribution? (C18-1)
Copied to clipboard
| Challenge: | Authorship attribution uses all information representing content and style whereas stylometry is robust in cross-domain settings. |
| Approach: | They analyze the role of syntax and lexical words in representing style . they show that syntax may be helpful for cross-genre attribution . |
| Outcome: | The proposed model may not be effective alone and needs to be combined with other robust models. |
Scalable Syntax-Aware Language Models Using Knowledge Distillation (P19-1)
Copied to clipboard
| Challenge: | Prior work has shown that syntactic neural language models learn from small amounts of training data more effectively than sequential models. |
| Approach: | They propose a knowledge distillation technique that transfers knowledge from a syntactic language model trained on a small corpus to an LSTM language model and enables it to develop a more structurally sensitive representation of the larger training data. |
| Outcome: | The proposed method improves on baseline syntactic evaluations on LSTMs with a higher level of accuracy than previous methods. |